The present invention relates to capturing and archiving computer network traffic. Networks allowing computer users to communicate and share information with one another are ubiquitous in business, government, educational institutions, and homes. Computers communicate with one another through small and large local area networks (LANs) that may be wireless or based on hard-wired technology such as Ethernet or fiber optics. Most local networks have the ability to communicate with other networks through wide area networks (WANs). The interconnectivity of these various networks ultimately enables the sharing of information throughout the world via the Internet. In addition to traditional computers, other information sharing devices may interact with these networks, including cellular telephones, personal digital assistants (PDAs) and other devices whose functionality may be enhanced by communication with other persons, devices, or systems.
The constant increase in the volume of information exchanged through networks has made network management both more important and more difficult. Enforcement of security, audit, policy compliance, network performance and use analysis policies, as well as data forensics investigations and general management of a network may require access to prior network traffic. Traditional storage systems, generally based on magnetic hard disk drive technology, have not been able to keep pace with expanding network traffic loads due to speed and storage capacity limitations. Use of arrays of multiple hard disks, increases speed and capacity but even the largest arrays based on traditional operating system and network protocol technologies lack the ability to monolithically capture and archive all traffic over a large network. Capture and archive systems based on current technologies also become part of the network in which they function, rendering them vulnerable to covert attacks or “hacking” and thus limiting their security and usefulness as forensic and analytical tools.
To overcome these limitations, a robust network packet capture and archiving system must utilize the maximum capabilities of the latest hardware technologies and must also avoid the bottlenecks inherent in current technologies. Using multiple gigabit Ethernet connections, arrays of large hard disk drives, and software that by-passes traditional bottlenecks by more direct communication with the various devices, it is possible to achieve packet capture and archiving on a scale capable of handling the traffic of the largest networks.